1. Technical Field of the Invention
This invention most generally relates to data transfer and communication network. In particular, the present invention relates to a device and system for high bandwidth data transfer using fiber optics.
2. Background of the Invention
Technological advancements have dramatically increased the capabilities and possibilities of computing electronics. The increased bandwidth and data transfer rates have resulted in commercial innovation and scientific advancements in many fields. However, data transfer continues to be a bottleneck. Present network communications that connect a multiple of nodes suffers from inefficiencies that bog down high-speed data communications.
A driving factor leading to ever increasing demands for faster data transfer rates is the need to do tasks that are more complex, requiring multiple computing nodes to cooperate. Digital signal processing, image analysis, and communications technology all require a greater bandwidth. The demand for increased data transfer capability and greater bandwidth translates into increases in both the speed of the data transfer, and the amount of data that is transferred per unit time.
Latency is defined as the amount of time it takes for data to be sent from a source node to a destination node. One of the key impediments to significantly increasing the speed with which communications devices can communicate with one another is the very limited capability of existing systems to transfer data in parallel. A significant source of latency is the need for reading and interpreting the address of each data packet, whether or not the data is intended for that particular device. The process of reading and interpreting packet destination addresses is done at each device in the network, and results in a dramatic limitation in the speed of data transfer within the network.
In general, the problems associated with data transfer on a system network can be alleviated by increasing the number of data transfer lines and transferring the data in parallel, and/or increasing the transmission speed. But, there are limitations to the number of I/O lines, such as spacing and size requirements, noise problems, reliability of connectors, and the power required to drive multiple lines off-chip. Increasing the transmission speed also has some limitations, as increasing the speed also increases power requirements, introduces timing skew problems across a channel, and usually requires more exotic processing than is standard practice. Combining higher clock speeds and more I/O connections in order to increase bandwidth is exceedingly difficult and impractical using electronics alone. Thus, using traditional technology there is a practical limitation in traditional data transfer notions, and the associated problems that are well known in the art.
A local area network (LAN) is a means of interconnecting multiple computers. A variety of standards exist, with the most popular perhaps being the family of “Ethernet” standards (ANSI/IEEE standard 802.3 and others). Like a computer system bus, an Ethernet network consists of a shared medium (coaxial cable) over which all data is transferred. LAN's typically have lower bandwidth than system busses, but allow nodes to communicate at larger distances. Several Ethernet standards exist, with data transfer rates of 10 Mbps (millions of bits per second), 100 Mbps and 1 Gbps. Nodes may be separated by distances of up to 100 meters using Ethernet, which is much greater than system bus dimensions that are typically a fraction of a meter.
Local area networks such as Ethernet carry the bulk of the data transfer between systems and individual users. Ethernet, in fact, is a very widely used communications standard for most local area networks. In general, there are three types of LAN networks, namely the linear bus, ring, and star.
The linear bus network is shown in FIG. 1, where a plurality of nodes 10 are interconnected along a line 5. The parallel node connections are effected through direct connection or attenuation taps. Unfortunately, fiber optics are not easily amenable to a parallel interface and using fiber optics for linear bus networks is difficult to implement. In addition, the parallel structure requires extensive addressing and contention remedies which decreases efficiency.
One of the more common original network topologies is the ring network shown in FIG. 2A. The ring topology enables communication around a ring serially through each of a number of nodes 20. Each user or node 20 transmits data messages serially around the ring in a clockwise or counterclockwise direction by some form medium of transmission 30 such as free-space optics using mirrors, or through direct connections such as fiber optics.
The vast majority of Fiber Distributed Data Interface (FDDI) rings transmit clockwise and counterclockwise simultaneously as illustrated in FIG. 2A. This bi-directional transmission technique is used to assure that data transmission will continue around the ring in cases where a single node becomes inoperable. However, when two nodes on either side of a working node become inoperable, communications from that working node will cease.
A drawback of the ring topology is the data transmission delay or latency incurred as the message is passed through each node. Local area network systems are typically limited to twenty-five nodes or less in an effort to limit accumulated system latency. Large systems are typically partitioned into several rings in an effort to manage system latency.
FIG. 2B illustrates one embodiment of a multi-Ring LAN system using partitioning to manage latency effects. Reducing the number of nodes within the ring reduces latency within each ring. The intersecting B Node 40 provides a data communications “bridge” between each ring 50, 60, thereby enabling communication between the rings 50, 60. As shown in FIG. 2B, for a bi-directional system, the maximum amount of delays between any two nodes within a single ring 50 or 60 is three node delays. The maximum amount of node delay between Node A 70 of the first ring 50 and Node B 80 of the second ring 60 is seven node delays.
A further embodiment showing a three-ring Ethernet system is illustrated in FIG. 2C. The “B” nodes 100 provide a bridge between rings 110, 120, and 130. Again, the latency within each ring is improved by reducing the number of nodes within each ring. However, as the number of rings increases, the latency between outer rings increases. FIG. 2C illustrates eleven node delays between node NA 140 and Node NB 150 of the outer rings 110 and 130 respectively.
Demand for even higher speed data communications however has driven network design beyond just increasing the interconnect speeds to other network topologies in an effort to improve system latency and bandwidth.
The star network topology has emerged as a topology that is especially well suited to enable point to point communications with low latency. FIG. 3A illustrates one embodiment of a networked system utilizing a star topology that interconnects a plurality of nodes 210. In this embodiment, data transfer occurs through the central or center node 220. The advantage to this topology is that only a single node delay is incurred between nodes within the star network. However, a disadvantage of the star topology is the requirement that all data must be processed by the central node 220 in order to ascertain the destination address. The data packet includes information in the header, such as destination address, that is read by the central node each time a packet encounters a central node. The processing time for reading each packet contributes to overall latency.
For example, a data message from node 1 would travel to the central node 220. The central node reads the header of the data for the destination address and transfers the packet to node 5 as illustrated in FIG. 3A. A single node delay through the central node 220 is thus incurred for each data transfer within the star network.
FIG. 3B illustrates an embodiment of a three star network topology 250 where nodes “B” 300, 305 provides a bridge between star networks. In this embodiment, the maximum amount of delay between any two nodes is five node delays. For example, a data message from node NA would travel to the central node A 280 of the outer star, then through the bridge node B 300 to the center node 220 and bridged again at node B 305 by the middle star, then carried through to center node B 290 of the other outer ring before reaching its destination NB. The star network topology exhibits lower latency than the ring topology. If the bridge nodes 300, 305 are omitted—and center nodes 270, 280, 290 connected directly, the configuration is termed a “switch fabric” or “switch network”.
An advantage of a switched network is that one pair of nodes can communicate simultaneously with a second pair of nodes, as long as there is no contention. Switched fabrics can also scale to hundreds or thousands of nodes, since all connections are point-to-point and capacitance does not grow linearly with the number of nodes. One problem with switched networks is that some contention may still exist in the network when more than one pair of nodes tries to communicate, since they both may need to use the same switch-to-switch link along their paths. An ideal switched network is called a “crossbar” and consists of a single large switch that connects directly to all nodes in the system, and can provide contention-free communications among them. Unfortunately, a full crossbar is difficult to manufacture and implement.
A number of switched fabric standards exist now or have been proposed to replace system busses, including Myrinet, RaceWay, the Scalable Coherent Interconnect (SCI), RapidIO, and InfiniBand. These are sometimes called “system area networks” (SANs) or “storage area networks” if used to connect processors to disk drives. Switch fabric standards are also in widespread use for local area networks, including switched Ethernet, Myrinet, and Asynchronous Transfer Mode (ATM).
Data transfer protocols are established by a number of standards. These standards all employ standard ways of formatting data in discrete chunks called frames or packets. The packet or frame establishes the format of the data and the various fields and headers are encapsulated and transmitted across a network. A frame or packet usually includes a destination address, control bits for flow control, the data or payload, and error checking in the form of cyclic redundancy checks (CRC) codes or an error correcting code (ECC), as well as headers and trailers to identify the beginning and end of the packet. As information is communicated between devices or systems, the address information is checked by each device or system in the network, and eventually the device of interest receives the data.
Whether transferring data within a circuit or connecting system-to-system, the limited bandwidth of conventional hardware does not satisfy the marketplace. For high data rate transmissions, fiber optics transmits data at Gigabit data rates. Fiber optic communication systems allow information to be transmitted by means of binary digital transmission. The data or information that is to be transmitted is converted into a stream of light pulses, wherein the presence of a pulse corresponds to the transmission of a binary “one,” and the absence of light corresponds to the transmission of a binary “zero.” An optical receiver is used to convert the stream of light pulses into an electrical signal that is processed to determine the transmitted information. Fiber-optic standards for LANs exist and are in widespread use today, including the FDDI, FibreChannel and several ATM physical layers.
Some attempts have been made to increase bandwidth and data transfer efficiency. The use of smart pixels to provide the required interconnection has been developed. “Smart Pixel” refers to the optical interconnection for digital computing systems such as switching systems and parallel-processor systems. For example, large numbers of optical transmitters and receivers are directly integrated with semiconductor electronic processing elements. The integrated optoelectronic circuits have several benefits, including efficiency of design.
Passive optical technology is used to provide point-to-point high bandwidth connectivity and nothing else. The underlying architecture does not support broadcast channels, one-to-many communications over a single channel, or one-to-all communications over a single channel, simultaneous many-to-many communications over multiple channels. The architecture simply implements multiple passive point-to-point interconnects with no broadcasting. Since this architecture cannot support broadcasting it will have limited use in computing and communications systems which require efficient broadcasting.
Furthermore, the passive optical architecture has power limitations as the number of receivers increases, because the architecture does not allow for the regeneration of optical signals. A fraction of each optical signal is delivered to each photodetector receiver through the use of partially reflective micromirrors. This free-space technique allows an optical signal to be delivered to a small number of receivers, but it cannot be used to interconnect a large number of receivers since the original optical signal can only pass through a limited number of partially reflective mirrors before the signal is lost.
Although some researchers have demonstrated Terabits/s serial connection, the methodology is overly complex and the price and size of these systems is impractical for system area networks. Recent innovations have permitted wavelength division multiplexing (WDM) systems to increase their bandwidth considerably, however, this is primarily a telecommunications, wide-area networking (WAN) solution. WDM systems are still relatively large and expensive, but compared to laying new fibers across the country the cost of the transmitters and receivers seems insignificant. For a local area network (LAN) or system area networks (SANs), WDM is generally cost-prohibitive and often will not meet form-fit-factors requirements. For LANs/SANs, the problems preventing effective wide bandwidth are: connector size and reliability, channel skew, wire impedance, and power dissipation.
Overall, the complexity and cost of the prior systems have prevented large-scale integration. Thus, there is a need for increased system bandwidth through both increased data rates and improved mechanical and electrical interconnects.
What is needed is a means for reducing the latency so that it is not a significant factor in limiting data transfer. In other words, what is needed is a way of transferring data from one node in a network to any other node in the network in a bit-parallel manner in such a way that each intervening node that touches the data (whether switch or network interface controller—NIC) minimizes the time required to process data through. In one case, the switch/device should act like wire or fiber and require no processing. What is needed is a way of resolving this address interpretation problem that eliminates the delay associated with the transfer of data. What is needed is a uniform device that can be used as both NIC and switch so that the switching function is essentially free and the NIC function is inexpensive. What is needed is a device that does not increase message latency by requiring packet loss checks and frequent retransmission of packets when contention occurs. Ideally, what is needed is a network with wide channels, fast links, small and reliable connectors, low power, low latency, and minimal impact on higher-level communication protocols. From a practical point of view, these features must be offered as a cost-effective solution.